The Department of Correctional Services makes permanent appointments to the rank of sergeant by means of competitive examination. In 1974, in a suit brought by black and hispanic corrections officers, Judge Lasker found that the test from which promotions to sergeant were being made at that time was discriminatory. Kirkland v. New York State Department of Correctional Services, 374 F. Supp. 1361 (S.D.N.Y.1974). We affirmed, 520 F.2d 420 (2d Cir. 1975). In our opinion, we spelled out the relief appropriate, which included the administration of a new, non-discriminatory test to be "validated in accordance with the E.E.O.C. Guidelines on Employment Selection Procedures," id. at 426 and we ordered that "the new testing procedures be validated by means of empirical criterion-related validation techniques if feasible." Id. at 431.

Pursuant to our directives, the Department developed Test No. 36-435. This examination consists of two parts: a written multiple-choice portion in which test-takers are asked to answer questions relating to the work of a corrections officer; and a set of "performance ratings" made by the test-taker's departmental superiors who have observed the candidate's on-the-job performance. The "performance ratings" are given in four major areas, and the candidates are assigned a numerical grade depending on how closely their performance meets, in the rater's view, the standards set out in paragraph-long descriptions of varying levels of job competence. For example, when rating an applicant's record in "Taking effective positive direct action in security situations," one of the four major areas graded, the standards range from:

Is alert and active in searching out and preparing for potential incidents. Acts efficiently and positively in taking charge at the scene of a disturbance . . .

to

He misappraises the security implications of situations. Does not take charge in situations which clearly call for it . . .

When a grader gives a particularly high or low grade (the examples given above represent opposite ends of the spectrum), he must provide additional explanation. Each applicant is graded by two different evaluators in the performance portion of Test No. 36-435, and if the scores assigned differ significantly, the two graders must explain, in writing, the reasons behind this difference. An applicant can appeal the score he receives in the performance portion of the exam, and if he does so, two other graders will be assigned to perform a de novo evaluation.

In December, 1978, Test No. 36-435 was given to approximately 2,300 applicants for the rank of sergeant. Pursuant to the district court's decree, a "validation study" was carried out in order to determine if the test would function accurately as a non-discriminatory predictor of on-the-job performance. The methodology employed, which in this respect is not challenged,1 was to compare the raw scores of applicants grouped by racial identity on the performance portions of the exam, to their raw scores on the entire exam, in which the written multiple-choice exam results figured. A disparity of, on the average, 268 points out of a total of 8,830 was discovered. This figure represents the fact that, on average, a black applicant's raw score on the total exam was 268 points lower than that of a white applicant, in a case where both black and white applicants had scored equally well in the performance rating portion of the exam. In order to "validate" Test No. 36-435, the Department re-scored exam results by adding 250 points to the raw score of every minority group applicant.

Intervenors, who are non-minority test-takers, were granted leave by the district court to file a complaint under 42 U.S.C. §§ 1981 and 1983. They argue that the 250-point bonus is tantamount to a quota illegal under federal and state law and that the test violates New York's Civil Service laws. Judge Lasker granted the motion for summary judgment against intervenors made by plaintiffs and defendants.

We affirm. The intervenors raised no triable issues of material fact regarding Test No. 36-435, and we agree with the district court's conclusions of law.

Appellants are entirely misguided in arguing that the 250-point differential is a quota,2 for it does not require that a minimum number of sergeant appointments be given to any members of a minority group. To the extent that appellants' argument is directed toward the fundamental fairness of the 250-point adjustment, we note that the district court found that the differential was necessary to prevent future discrimination of the kind found to have existed earlier in this case. Without the 250-point correction, the new test like its predecessor would not be "valid", since it would not serve as a race-neutral predictor of on-the-job performance. Contrary to appellants' interpretation, the E.E.O.C. guidelines recognize the possibility that a point correction to raw scores should be made in some situations, i. e., "to assure compatibility between the probability of successful job performance and the probability of being selected."3 29 C.F.R. § 1607.14(B)(8)(d).

Appellants also argue that the 250-point differential is "reverse discrimination" in violation of part of our holding in the earlier appeal in this case, 520 F.2d 420 (2d Cir. 1979). In that opinion we said that "no one should be 'bumped' from a preferred position on the eligibility list solely because of his race." Id. at 429. Although we do not discount the continuing validity of this statement, we observe that its context demonstrates that it was directed primarily to the question of minority group quotas for promotions. Kirkland forbade such quotas and, as we have said above, the steps taken by the Department at issue in this case do not constitute de jure or de facto quotas. This program does not bump white candidates because of their race but rather re-ranks their predicted performance as estimated by the combined test score and job performance ratings. Finally, our previous opinion approved a broad range of affirmative action relief, including quotas in interim appointments, id. at 429-430, and made a distinction between quotas in initial hiring decisions and in promotions.

Of course, the 250-point differential does operate to favor minority test-takers if one assumes that the multiple-choice written portion of the exam is a truly objective measure of qualities important to success as a corrections officer. But even if "(s)ome non-minority applicants near the cut-off scores of appointment will be effectively denied employment owing to the elevation of scores of minority candidates," appellants have not cited us to any persuasive authority in federal law that makes such denial illegal. In light of the importance of non-discriminatory hiring in correctional institutions, where a high percentage of the inmates are members of minority groups, the lack of any such prohibition is easily understood.

Moreover, the issue of the 250-point differential is settled by the law of the case doctrine. Our earlier Kirkland decision requires any new test "to be validated by means of empirical criterion-related validation techniques if feasible." 520 F.2d at 431. By requiring the use of such techniques we, in effect, foreclosed the argument made now by intervenors. Criterion validation requires the abstracting of job performance characteristics and the rating of employees as to how they measure up to these characteristics. These ratings can then be compared to the performance of applicants who have taken a written exam. If significant variation correlating to racial identity is found in such a comparison, the test fails to be criterion valid. In this case, the abstracting of job performance characteristics and the rating of applicants were performed as part of the examination later validated. Appellants' argument that the 250-point differential constitutes tampering with the results of a valid test thus puts the cart before the horse; the results of the written portion of the exam are not valid until they have been compared to job performance criteria. As the E.E.O.C. guidelines recognize, once the results of such a comparison indicate a discrepancy, the responsible authorities have a choice. They can return to the drawing board and design a new test meant to correlate to job performance ratings; or they can adjust the test to bring the results into conformity with those ratings. See 29 C.F.R. § 1607.14(8)(d). In this case, they chose the latter course of action, an alternative clearly foreseen by our prior opinion.

Appellants' second major group of arguments assert that New York's Civil Service laws and policies prohibit a test that, like No. 36-435, emphasizes subjective job performance evaluation over more traditionally objective varieties of competitive examination, such as written multiple-choice questions. The primacy given the job performance portion of the test cannot be disputed. Indeed, since the effect of the 250-point differential is to bring the average minority applicant's total raw score up to the level of non-minority test-takers with equal job performance ratings, it can be argued that at least insofar as deviations from the average are disregarded the results of the written portion of Test No. 36-435 are entirely superfluous when minority applicants are involved. The question that must be answered is whether or not New York law permits appointments to be made on the basis of job performance evaluations. We conclude that it does.

New York law does not reject all means of making civil service appointments differing from the norm of competitive written examinations. The constitutional provision in question, Article V, § 6, reads in pertinent part:

Appointments and promotions in the civil service of the state and all of the civil divisions thereof, including cities and villages, shall be made according to merit and fitness to be ascertained, as far as practicable, by examination which, as far as practicable, shall be competitive . . .

Interpreting this provision, New York courts have made it clear that the use of the terms "merit and fitness" and "as far as practicable" justify interviews as permissible departures from objective written examinations. Walker v. Board of Examiners, 22 Misc. 2d 345, 204 N.Y.S.2d 432 (Sup.Ct. N.Y.Co.1957), aff'd mem., 7 A.D.2d 968, 183 N.Y.S.2d 990 (1st Dept. 1959). Consideration of on-the-job performance as a factor in granting a teacher's license was approved in Sloat v. Board of Examiners, 274 N.Y. 367, 9 N.E.2d 12 (Ct.App.1937). In Young v. Trussel, 42 Misc. 2d 108, 247 N.Y.S.2d 603 (Sup.Ct. N.Y.Co.1964), the court approved a procedure whereby applicants for the post of psychologist were not only given a written examination but were assigned a "key rating" score based on certain numerical values attached to past experiential and educational qualifications. In Shanley v. Leonard, 87 Misc. 2d 915, 386 N.Y.S.2d 951 (Sup.Ct. Nassau Co.1976), the court approved a system whereby promotions were made by written exam supplemented by scoring differentials giving candidates additional credit for past departmental commendations. See also Bobrowich v. Poston, 52 A.D.2d 976, 383 N.Y.S.2d 113 (3d Dept. 1976); Chance v. Board of Education, 496 F.2d 820, 824 (2d Cir. 1974).

Appellants rely on two cases which invalidated, respectively, the 1961 and 1963 exams given to state police officers for promotion to police sergeant. Donohue v. Cornelius, 39 Misc. 2d 539, 241 N.Y.S.2d 683 (Sup.Ct. Albany Co.1963); Donohue v. Cornelius, 22 A.D.2d 1000, 254 N.Y.S.2d 941 (3d Dept. 1964). Both tests included a written component, a performance evaluation component, and an oral exam. The first Donohue opinion explained:

An examination cannot be classed as competitive unless it conforms to measures or standards which are sufficiently objective to be capable of being challenged and reviewed, when necessary, by other examiners of equal ability and experience.

39 Misc.2d at 541-542, 241 N.Y.S.2d at 686. The court noted, in invalidating the 1961 exam, that the oral portion of the exam included questions about the applicant's subjective beliefs and attitudes. The second Donohue opinion pinpointed as objectionable the use of performance ratings that, instead of being aggregates accumulated over time as part of a service record, were arrived at "specifically for purposes of the examination by a rating board guided only by general standards." 22 A.D.2d at 1001, 254 N.Y.S.2d at 943.

We think both cases are distinguishable from the case at bar. No candidates taking Test No. 36-435 were asked questions regarding their personal beliefs or attitudes. And although the performance ratings were not part of an ongoing service record, the standards used by the raters were not "general standards," as prohibited by Donohue II, but specific and detailed ones, each running to a paragraph of terse, descriptive prose.

In our view, the key to determining whether an examination is objective enough to qualify as within New York's civil service law was stated in that portion of the Donohue I opinion quoted above, and in Fink v. Finnegan, 270 N.Y. 356, 362, 1 N.E.2d 462 (Ct.App.1936). The guiding principle established in these cases is the reviewability of the exam. In the case of Test No. 36-435, officers who perform the evaluations must provide written explanations for high and low ratings, and there is a provision for written explanations when the ratings assigned a candidate by his two evaluators differ widely. Where the raters do not disagree, no explanation is required; in such cases the very degree of agreement provides a basis for sustaining the objectivity of the rating. The correlation between the values assigned by the two raters was .80 out of a possible 1.00, a high enough figure to support the district court's conclusion that the ratings were an objective measure of performance. Moreover, the testing procedures allow for appeals, including a de novo second rating by two officers who did not participate in the initial rating, and, of course, an applicant may bring an Article 78 proceeding in state court to obtain judicial review.

Additionally, we note that the use of on-the-job performance ratings as the principal method of making "merit and fitness" appointments to civil service positions is considerably more defensible when the appointment at issue is a promotion rather than an initial hiring. Cf. Kirkland, supra, at 429 (distinction between initial hirings and promotions in the context of quotas). Where, as here, each of the test-takers has an on-the-job history that can be evaluated, and where the position he seeks is one requiring qualities similar to those already displayed in the performance of his duties, the argument for reliance on performance ratings is stronger than if this were a case involving applicants for initial appointments.

Finally, appellants challenge the validity of the 250-point differential under state law. Appellees seek to justify the differential under state law by directing our attention to 4 N.Y.C.R.R. § 67.1(g), which allows for differential weighing of test components "for candidate subgroups, if scores of such subgroups are shown to differ substantially in their relation to measures of performance." Appellants respond that a variation of approximately 250 out of 8,830 points is not "substantial," and that in any event the regulation is inconsistent with Article V, § 6 of the New York Constitution, quoted supra, if the regulation is interpreted to allow additions to the scores of members of racial subgroups. We hesitate to say whether, as a matter of law, an approximately 3% variation is "substantial," because the answer may well depend on how test scores bunch together; for example, if most scores fell within 500 points of each other, a 250-point difference would clearly be quite substantial.

We see no need, however, to reach the question of the compatibility of the 250-point addition with the New York Constitution. In this case, a finding of discrimination unconstitutional under the United States Constitution has been upheld on prior appeal, and the power of the district court to fashion a remedy is a matter of federal law under the supremacy clause.

Affirmed.